Handling API Responses, Status Codes, and JSON Data in Python

Course: Python Training

Topic: Handling API Responses, Status Codes, and JSON Data

Course Link: Python Training Course at JustAcademy

Demo Link: Register for a Course Demo

Official Website: JustAcademy

1. Introduction to API Responses

An API (Application Programming Interface) allows two software applications to communicate and exchange information. Python applications use APIs to retrieve weather information, access user accounts, fetch product details, process payments, and integrate with external services.

When a Python program sends an HTTP request to an API, the server returns an HTTP response. This response usually contains a status code, headers, and sometimes a body containing JSON or another data format.

Handling API responses correctly involves checking whether the request succeeded, interpreting the status code, parsing the response body, validating the returned data, and handling errors appropriately.

Example of an API Response

{
    "userId": 12,
    "name": "Aarav Sharma",
    "email": "[email protected]"
}

This JSON object could represent a user record returned by an API. Python can convert the JSON data into a dictionary and access fields such as name and email.

2. Learning Objectives

• Understand the structure of an HTTP response.
• Send API requests using Python's requests library.
• Interpret common HTTP status codes.
• Parse JSON responses into Python objects.
• Handle network errors, HTTP errors, and invalid JSON.
• Validate API response data before using it.
• Work with response headers, timeouts, and query parameters.
• Build reusable functions for API communication.
• Apply API response handling in practical Python projects.

3. Understanding the HTTP Response Structure

An HTTP response provides information about how a server processed a request.

Not every response contains a body. For example, 204 No Content indicates that the request succeeded without returning response content. Applications should not assume that every successful response contains JSON.

4. Installing and Importing the Requests Library

The requests library is a popular Python package for making HTTP requests. It supports GET, POST, PUT, PATCH, and DELETE operations.

4.1 Install Requests

python -m pip install requests

4.2 Import the Library

import requests

For projects using a virtual environment, activate it before installing dependencies. Document external packages in a file such as requirements.txt when appropriate.

5. Sending a GET Request

A GET request is commonly used to retrieve information from an API without requesting a change to the resource.

import requests

url = "https://jsonplaceholder.typicode.com/users/1"

try:
    response = requests.get(url, timeout=10)

    print("Status code:", response.status_code)
    print("Response URL:", response.url)
    print("Response body:", response.text)

except requests.exceptions.RequestException as error:
    print("Request failed:", error)

This example uses a public demonstration API. The timeout limits how long the client waits for the request, while the exception handler catches request-related problems such as connection failures and timeouts.

Important Response Attributes

6. Understanding HTTP Status Codes

HTTP status codes are three-digit numbers grouped into categories. They help clients determine whether a request succeeded, requires further action, or failed.

7. Common HTTP Status Codes

7.1 Successful Status Codes

7.2 Client Error Status Codes

7.3 Server Error Status Codes

Status codes describe the HTTP-level result. An API may also include application-specific error codes and messages in its response body. Check the service documentation to understand its expected behavior.

8. Parsing JSON Responses in Python

JSON (JavaScript Object Notation) is a lightweight data format widely used to exchange information between clients and servers. It supports objects, arrays, strings, numbers, Boolean values, and null values.

The requests library provides the response.json() method to decode JSON response content into Python objects. The resulting object may be a dictionary, list, string, number, Boolean, or None, depending on the JSON content.

8.1 Convert a JSON String into a Dictionary

import json

json_data = '''
{
    "id": 101,
    "name": "Aarav",
    "active": true,
    "score": 89.5
}
'''

data = json.loads(json_data)

print(data["id"])
print(data["name"])
print(data["active"])
print(data["score"])

Output:

101
Aarav
True
89.5

The json.loads() function converts a JSON string into a Python object. JSON's true, false, and null values become Python's True, False, and None, respectively.

8.2 Parse JSON Directly from an API

import requests

url = "https://jsonplaceholder.typicode.com/users/1"

try:
    response = requests.get(url, timeout=10)
    response.raise_for_status()

    user = response.json()

    print("User ID:", user["id"])
    print("Name:", user["name"])
    print("Email:", user["email"])

except requests.exceptions.HTTPError as error:
    print("HTTP error:", error)
except requests.exceptions.JSONDecodeError:
    print("The response does not contain valid JSON.")
except requests.exceptions.RequestException as error:
    print("API request failed:", error)
except (KeyError, TypeError) as error:
    print("Unexpected response structure:", error)

The code checks for HTTP errors before parsing the response. It also handles invalid JSON and unexpected fields. Adapt the expected fields to the API's documented response format.

9. Working with JSON Arrays and Nested Objects

Some APIs return lists of records rather than a single object. JSON arrays become Python lists, and JSON objects inside those arrays become dictionaries.

9.1 Process a List of Records

import json

json_data = '''
[
    {"id": 1, "name": "Aarav", "department": "IT"},
    {"id": 2, "name": "Meera", "department": "HR"},
    {"id": 3, "name": "Kabir", "department": "Finance"}
]
'''

employees = json.loads(json_data)

for employee in employees:
    print(employee["name"], "-", employee["department"])

Output:

Aarav - IT
Meera - HR
Kabir - Finance

9.2 Access Nested JSON Data

data = {
    "id": 25,
    "name": "Meera",
    "address": {
        "city": "Mumbai",
        "postal_code": "400001"
    }
}

print(data["address"]["city"])
print(data["address"]["postal_code"])

Nested objects can be accessed through consecutive dictionary keys. Direct indexing can raise KeyError when a key is missing or TypeError when a value has an unexpected type.

9.3 Safely Access Optional Fields

user = {
    "id": 25,
    "name": "Meera"
}

email = user.get("email", "Email not provided")
print(email)

The get() method is useful for optional fields. It does not replace validation when a field is required for the application to work correctly.

10. Checking Status Codes in Python

You can inspect response.status_code to determine the HTTP result and choose an appropriate next step. Do not assume every response is successful simply because a response object was returned.

10.1 Using Conditional Statements

import requests

url = "https://jsonplaceholder.typicode.com/users/1"

try:
    response = requests.get(url, timeout=10)

    if response.status_code == 200:
        print("Request successful")
        print(response.json())
    elif response.status_code == 404:
        print("Requested resource was not found")
    elif response.status_code == 429:
        print("Rate limit reached")
    elif 500 <= response.status_code <= 599:
        print("Server error:", response.status_code)
    else:
        print("HTTP status:", response.status_code)

except requests.exceptions.RequestException as error:
    print("Network or request error:", error)

This approach is useful when different status codes require different application behavior. The exact status codes and actions should follow the API documentation.

10.2 Using raise_for_status()

The raise_for_status() method raises an HTTPError for HTTP 4xx and 5xx responses. It does not raise an error for every non-200 response; for example, a 3xx redirect may be handled automatically by Requests.

import requests

try:
    response = requests.get(
        "https://jsonplaceholder.typicode.com/users/1",
        timeout=10
    )
    response.raise_for_status()
    data = response.json()
    print(data)

except requests.exceptions.HTTPError as error:
    print("HTTP error:", error)
except requests.exceptions.RequestException as error:
    print("Request error:", error)
except requests.exceptions.JSONDecodeError:
    print("Invalid JSON response.")

For applications requiring an exact status such as 200 or 201, check the code explicitly. A response can be successful at the HTTP level but still not meet the application's requirements.

11. Handling Invalid JSON and Empty Responses

An API might return malformed JSON, plain text, HTML, or an empty response. Calling response.json() without considering these possibilities can raise a decoding exception.

11.1 Handle JSON Decoding Errors

import requests

try:
    response = requests.get(
        "https://jsonplaceholder.typicode.com/users/1",
        timeout=10
    )
    response.raise_for_status()

    data = response.json()
    print(data)

except requests.exceptions.HTTPError as error:
    print("HTTP error:", error)
except requests.exceptions.JSONDecodeError:
    print("Could not decode the response as JSON.")
except requests.exceptions.RequestException as error:
    print("Request failed:", error)

11.2 Handle a No-Content Response

if response.status_code == 204 or not response.content:
    print("Request completed without response content.")
else:
    response.raise_for_status()
    data = response.json()

Use this logic after sending the request. A no-content response should be handled without attempting to parse an absent body as JSON.

11.3 Check the Content Type

content_type = response.headers.get("Content-Type", "")

if "application/json" in content_type.lower():
    data = response.json()
else:
    print("Expected JSON, received:", content_type)

The content type is a useful initial check, but it does not guarantee that the body contains valid JSON. Parse the body safely and validate the expected structure.

12. Handling Network Errors and Timeouts

Network problems can occur when a server is unavailable, a connection is refused, DNS resolution fails, or a request takes too long. Use timeouts so an application does not wait indefinitely for a response.

12.1 Configure a Timeout

import requests

try:
    response = requests.get(
        "https://jsonplaceholder.typicode.com/users",
        timeout=(3, 10)
    )
    response.raise_for_status()

    users = response.json()
    print("Users received:", len(users))

except requests.exceptions.Timeout:
    print("The request timed out.")
except requests.exceptions.RequestException as error:
    print("API request failed:", error)
except requests.exceptions.JSONDecodeError:
    print("The response was not valid JSON.")

The tuple (3, 10) sets a connection timeout of 3 seconds and a read timeout of 10 seconds. These values do not necessarily impose a strict total time limit on the entire response download.

13. Reading Response Headers

Response headers provide metadata about the server response. They may indicate content type, caching rules, rate limits, or when the response was generated.

import requests

response = requests.get(
    "https://jsonplaceholder.typicode.com/users/1",
    timeout=10
)

print("Status:", response.status_code)
print("Content-Type:", response.headers.get("Content-Type"))
print("Cache-Control:", response.headers.get("Cache-Control"))
print("Response Date:", response.headers.get("Date"))

Common headers include:

• Content-Type: Describes the response body's media type.
• Content-Length: May indicate the response body's size.
• Cache-Control: Provides caching directives.
• Date: Indicates the response date when supplied by the server.
• Retry-After: May indicate how long a client should wait before retrying.
• Link: May provide links to related resources or pagination pages.

Header availability and meaning depend on the server. Use the API documentation to determine which headers your application can rely on.

14. Sending Query Parameters

Query parameters allow clients to filter, search, sort, or paginate API results. Requests can construct the query string from a Python dictionary.

import requests

url = "https://jsonplaceholder.typicode.com/posts"
params = {
    "userId": 1,
    "_limit": 5
}

try:
    response = requests.get(url, params=params, timeout=10)
    response.raise_for_status()

    posts = response.json()

    print("Requested URL:", response.url)
    for post in posts:
        print(post["id"], post["title"])

except requests.exceptions.RequestException as error:
    print("Request failed:", error)
except requests.exceptions.JSONDecodeError:
    print("Unexpected JSON response.")
except (KeyError, TypeError) as error:
    print("Unexpected response structure:", error)

Using params is safer and more convenient than manually concatenating query strings because Requests handles URL encoding for supplied values.

15. Sending JSON Data to an API

Some APIs accept JSON request bodies for creating or updating resources. Requests provides the json parameter to serialize a Python dictionary or another JSON-compatible value.

import requests

url = "https://jsonplaceholder.typicode.com/posts"

payload = {
    "title": "Learning Python APIs",
    "body": "Practicing JSON requests and responses.",
    "userId": 1
}

try:
    response = requests.post(url, json=payload, timeout=10)
    response.raise_for_status()

    print("Status code:", response.status_code)

    if response.content:
        try:
            print("Response JSON:", response.json())
        except requests.exceptions.JSONDecodeError:
            print("Response body is not valid JSON.")
    else:
        print("No response content.")

except requests.exceptions.RequestException as error:
    print("Request failed:", error)

Many APIs return 201 Created after creating a resource, but the exact response depends on the service. A demonstration API may simulate the operation without permanently storing submitted data.

16. Validating JSON Response Data

Valid JSON does not necessarily mean that the response contains the fields, types, or values an application expects. Validate the structure before processing the information.

16.1 Check Required Fields and Types

def validate_user(data):
    if not isinstance(data, dict):
        raise ValueError("Expected a JSON object")

    required_fields = ["id", "name", "email"]

    for field in required_fields:
        if field not in data:
            raise ValueError(f"Missing required field: {field}")

    if not isinstance(data["id"], int):
        raise ValueError("User ID must be an integer")

    if not isinstance(data["name"], str):
        raise ValueError("Name must be a string")

    if not isinstance(data["email"], str):
        raise ValueError("Email must be a string")

    return data

16.2 Use the Validation Function

user_data = {
    "id": 10,
    "name": "Aarav",
    "email": "[email protected]"
}

try:
    valid_user = validate_user(user_data)
    print("Validated user:", valid_user)
except ValueError as error:
    print("Invalid API data:", error)

For more complex applications, consider a validation library or schema-based approach. Validation rules should reflect the API contract and the application's requirements.

17. Building a Reusable API Request Function

Reusable functions reduce duplicated code and make API handling more consistent. A central function can apply timeouts, check HTTP status, parse JSON, and report failures.

import requests

def get_json(url, params=None, timeout=10):
    try:
        response = requests.get(
            url,
            params=params,
            timeout=timeout
        )

        response.raise_for_status()

        if response.status_code == 204 or not response.content:
            return None

        return response.json()

    except requests.exceptions.HTTPError as error:
        print("HTTP error:", error)
    except requests.exceptions.JSONDecodeError:
        print("The server returned invalid JSON.")
    except requests.exceptions.Timeout:
        print("The API request timed out.")
    except requests.exceptions.RequestException as error:
        print("Request error:", error)

    return None


result = get_json(
    "https://jsonplaceholder.typicode.com/users/1"
)

if result is not None:
    print("API data:", result)

This function is suitable as a learning example. In a larger application, returning None for every failure can make it difficult to distinguish a valid empty result from an error. Consider raising custom exceptions or returning a structured result with a clear error-handling strategy.

18. Understanding Rate Limits and Retries

Some APIs restrict how many requests a client can make within a specific period. When a rate limit is exceeded, an API may return 429 Too Many Requests and include a Retry-After header.

Server errors such as 502, 503, or 504 may be temporary. Repeated requests should be limited and should follow the API's usage rules.

Best Practices for Retries

• Use a finite retry limit.
• Respect the API's Retry-After guidance where provided.
• Use backoff delays rather than sending repeated requests immediately.
• Retry only errors that may be temporary.
• Do not blindly retry operations that may create duplicate transactions or resources.
• Use idempotency keys where supported for sensitive operations such as payment creation.
• Follow documented API rate limits and authentication requirements.

19. API Authentication and Secure Data Handling

Some APIs require an API key, bearer token, or another authentication method. Credentials should be stored securely and must not be hard-coded into source code.

Example Using an Authorization Header

import os
import requests

token = os.getenv("API_ACCESS_TOKEN")

if not token:
    raise RuntimeError("API_ACCESS_TOKEN is not configured")

headers = {
    "Authorization": f"Bearer {token}",
    "Accept": "application/json"
}

response = requests.get(
    "https://api.example.com/profile",
    headers=headers,
    timeout=10
)

response.raise_for_status()
data = response.json()

The URL above is a placeholder and must be replaced with an actual API endpoint. Set the token securely in the environment before running the example.

Security recommendations:

• Use HTTPS for API communication.
• Never publish access tokens, passwords, or private keys.
• Avoid printing authorization headers or sensitive response data in logs.
• Do not disable TLS certificate verification in normal application code.
• Store only the data the application actually needs.
• Rotate exposed credentials immediately.

20. Logging API Responses and Errors

Logging helps developers investigate failures and understand application behavior. Logs should provide enough detail for debugging without exposing secrets or sensitive user information.

import logging
import requests

logging.basicConfig(level=logging.INFO)

url = "https://jsonplaceholder.typicode.com/users/1"

try:
    response = requests.get(url, timeout=10)
    logging.info("API returned status %s", response.status_code)
    response.raise_for_status()

    data = response.json()
    logging.info("API data retrieved successfully")

except requests.exceptions.RequestException as error:
    logging.error("API request failed: %s", error)
except requests.exceptions.JSONDecodeError:
    logging.error("API response could not be parsed as JSON")

For production systems, use structured logs, appropriate log levels, and a central logging system when needed. Avoid logging complete response bodies if they may contain personal information, authentication data, or other sensitive content.

21. Common Mistakes and Troubleshooting

22. Practical Project: Weather API Response Handler

Project objective: Create a Python program that requests weather information from an API, checks the HTTP response, parses JSON, validates required fields, and displays a readable summary.

Required Features

• Read the API endpoint and required configuration securely.
• Send an HTTP GET request with a timeout.
• Check the HTTP status code.
• Parse JSON safely.
• Validate required fields before accessing them.
• Handle connection errors, timeouts, HTTP errors, and invalid JSON.
• Display a clear message when data is unavailable.
• Keep credentials out of the source code.

Example Response Processing

def display_weather(data):
    if not isinstance(data, dict):
        raise ValueError("Expected a JSON object")

    required = ["city", "temperature", "condition"]

    for key in required:
        if key not in data:
            raise ValueError(f"Missing field: {key}")

    print("City:", data["city"])
    print("Temperature:", data["temperature"])
    print("Condition:", data["condition"])


sample_weather = {
    "city": "Mumbai",
    "temperature": 29,
    "condition": "Cloudy"
}

display_weather(sample_weather)

This example demonstrates response validation using sample data. An actual weather service may use different field names or nested structures, so adapt the code to the selected provider's documented response schema.

23. Testing API Response Handling

Testing helps confirm that an application handles both expected responses and failures. Avoid depending entirely on live external services for automated tests because network conditions and service availability can change.

Test Cases to Include

• A valid response with the expected JSON object.
• A valid response containing a JSON array.
• A missing required field.
• An unexpected data type.
• An HTTP 404 or 500 response.
• An empty response body.
• Malformed JSON.
• A network connection error.
• A timeout.
• A rate-limit response.

Use mock responses or a test server to simulate failures predictably. Keep unit tests separate from integration tests that call an actual external API.

24. Best Practices for Handling API Responses

• Always set a reasonable timeout on network requests.
• Check HTTP status codes before relying on response data.
• Use raise_for_status() or explicit status checks as appropriate.
• Handle JSON decoding errors and empty response bodies.
• Validate response structure and required fields.
• Use params for query parameters and json for JSON request bodies.
• Follow the API's authentication, pagination, and rate-limit rules.
• Retry only appropriate failures and limit the number of attempts.
• Keep credentials and sensitive data out of logs and source code.
• Write tests for normal cases, edge cases, and failure scenarios.
• Document expected response formats and known limitations.
• Keep API handling logic reusable and easy to maintain.

25. Practical Exercises

1. Send a GET request to a public test API and print the status code.
2. Parse a JSON object and display selected fields.
3. Process a JSON array using a loop.
4. Write a function that handles HTTP 200, 404, and 500 responses.
5. Handle a response with invalid JSON.
6. Write a program that safely handles a 204 no-content response.
7. Send query parameters using a Python dictionary.
8. Submit a JSON payload to a demonstration endpoint.
9. Add a timeout and handle network exceptions.
10. Validate required fields before using returned data.
11. Create unit tests for successful and unsuccessful responses.
12. Build a weather response handler using sample data and a documented API schema.

26. Interview Questions and Answers

Q1. What is an API response?

Answer: An API response is the information returned by a server after processing an HTTP request. It may contain a status code, headers, and a response body.

Q2. What is JSON?

Answer: JSON is a lightweight data-interchange format used to represent structured data, including objects, arrays, strings, numbers, Boolean values, and null values.

Q3. How do you parse JSON in Python?

Answer: Use json.loads() to parse a JSON string or response.json() to decode JSON content from a Requests response.

Q4. What does HTTP 200 mean?

Answer: HTTP 200 OK indicates that the request succeeded. The response body depends on the endpoint and may contain JSON or another format.

Q5. What is the difference between HTTP 401 and 403?

Answer: A 401 response usually indicates that acceptable authentication credentials are missing or invalid. A 403 response indicates that the server refuses access, often because the client lacks permission.

Q6. What is HTTP 404?

Answer: HTTP 404 Not Found indicates that the requested resource could not be found.

Q7. What does raise_for_status() do?

Answer: It raises an HTTPError for HTTP 4xx and 5xx responses. It does not replace application-specific status checks.

Q8. Does successful JSON parsing mean the API request succeeded?

Answer: No. A server can return valid JSON containing error details with an unsuccessful HTTP status. Check the status code separately.

Q9. Why should API requests use timeouts?

Answer: Timeouts prevent the program from waiting indefinitely when a connection or response is slow or unavailable.

Q10. What happens if response.json() receives invalid JSON?

Answer: It raises a JSON decoding exception. The application should catch the exception and handle the unexpected response appropriately.

Q11. How do you access a field in nested JSON?

Answer: For nested dictionaries, access the required keys in sequence, such as data["address"]["city"], after validating the structure.

Q12. What is the purpose of the Content-Type header?

Answer: It indicates the media type of the response body, such as application/json or text/html.

Q13. How should a Python program handle HTTP 429?

Answer: It should respect the API's rate limits, check any Retry-After guidance, and retry only according to a bounded retry strategy.

Q14. How can you protect API credentials?

Answer: Store them in environment variables or a secure secret-management service, avoid committing them to Git, and rotate credentials if they are exposed.

Q15. How can API error handling be tested?

Answer: Use unit tests and mocked responses to simulate successful responses, HTTP errors, invalid JSON, missing fields, timeouts, and connection failures.

27. Quick Revision Checklist

• Understand the components of an HTTP response.
• Know the major HTTP status-code categories.
• Use Requests to send API requests.
• Parse JSON into Python objects.
• Handle JSON objects, arrays, and nested fields.
• Check status codes before processing the response.
• Handle HTTP errors, timeouts, and connection failures.
• Account for empty responses and malformed JSON.
• Read headers and pass query parameters correctly.
• Validate required fields and data types.
• Protect API credentials and sensitive information.
• Write tests for successful responses and common failures.

28. Conclusion

Handling API responses, status codes, and JSON data is a fundamental skill for Python developers working with web services, automation, data collection, and application integrations. Reliable API code does more than send a request: it checks the result, parses and validates returned data, handles failures, and protects sensitive information.

Practice these techniques with public test APIs before applying them to real services. Build small projects, document expected response formats, and test failure scenarios so your applications remain understandable and reliable.

To continue learning, visit the Python Training Course at JustAcademy, explore the JustAcademy website, or register for a course demo.

29. Additional Learning Resource

For further technical reference, consult the official Requests Quickstart documentation for details about JSON response parsing, HTTP status codes, timeouts, and exception handling.